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DETAILED ACTION 
Claim Rejections - 35 USC § 103 
The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all 

obviousness rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set forth in 
section 102 of this title, if the differences between the subject matter sought to be patented and the prior art are 
such that the subject matter as a whole would have been obvious at the time the invention was made to a person 
having ordinary skill in the art to which said subject matter pertains. Patentability shall not be negatived by the 
manner in which the invention was made. 

Claims 1-6 are rejected under 35 U.S.C. 103(a) as being unpatentable over Fridrich 
(6,101,602) in view of Wong (6,504,941 B2). 

In reference to claims 1 and 4, Fridrich discloses a method and system for authenticating 
using a watermark (abstract). The method comprises generating digital watermarks electronic 
documents, where the owner of a document hides a digital watermark as proof of identity in the 
document (Fig. 3), prior to being hidden, the watermark being not only provided with the proof 
of identity id, but also at least with the hash value h(m) of the document (column 5 lines 59-61). 
Fridrich verifies (authenticates) ownership of the document by comparing the hash value of the 
document (column 6 line 65 to column 7 line 5). 

Although Fridrich discloses the overlaid pattern depends on the key, Fridrich does not 
expressly disclose a secret key for making the watermark visible, characterized in that, to verify 
the true authorship, reversibly embedded watermarks are removed again with the assistance of 
the secret keys in order to restore the document to its original state, i.e., to check it on the basis 
of its hash values. 

Wong discloses a method and system that provides an invisible watermark that may be 
used in public key or secret key watermark systems (abstract). The system of Wong disclose a 
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secret key for making the watermark visible, characterized in that, to verify the true authorship, 
reversibly embedded watermarks are removed again with the assistance of the secret keys in 
order to restore the document to its original state, i.e., to check it on the basis of its hash values 
(Fig. 10 A in combination with column 2 line 59 to column 3 line 9). 

At the time the invention was made, it would have been obvious to a person of ordinary 
skill in the art to associate a user key with a watermark as in Wong in the system of Fidrich. One 
of ordinary skill in the art would have been motivated to do this because it may be used as a 
method of ownership verification so that the desired watermark can only be extracted from a 
watermarked image with the appropriate user key (Wong column 1 lines 50-64). 

In reference to claims 2 and 5, characterized in that, prior being hidden, the digital 
watermark is not only provided with the proof of identity id, also with an authentic time stamp, 
which, besides the time value t, also contains at least the hash value of the document, and, in 
addition, defines the embedding sequence (column 6 line 65 to column 7 line 11). 

In reference to claim 3, wherein embedded watermarks are removed, preferably under 
consideration of the embedding sequence, and the hash value of the thus created document 
subsequently generated, which is compared to the individual hash values in the different 
watermarks in order to determine the original owner (column 5 lines 59-65). 

In reference to claim 6, wherein the authentic time stamp defines an embedding 
sequence. Fridrich discloses the including the time stamp in the watermark (column 7 lines 1-5), 
therefore defining the embedding sequence because the time affects that watermark that is 
embedded. 
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Claims 7-8 are rejected under 35 U.S.C. 103(a) as being unpatentable over Fridrich and 
Wong as applied to claim 4 above, and further in view of Rhoads et al (6,636,615 Bl). 

Fridrich discloses using hash values to associate the digital watermark with a specific 
image and therefore use the hash value to determine the original owner (column 5 line 59 to 
column 6 line 3). 

However neither Fridrich nor Wong disclose using multiple watermarks. 

Rhoads discloses embedding several watermarks into the same image (Fig. 4 and column 
5 lines 15-46). It follows that to restore the image to the original state all the different 
watermarks would have to be removed using the method disclosed by Wong. 

At the time the invention was made, it would have been obvious to a person of ordinary 
skill in the art to embed multiple watermarks as disclosed by Rhoads in the system of Fridrich. 
One of ordinary skill in the art would have been motivated to do this because multiple 
watermarks can be used to convey multiple sets of information. 

In reference to claim 8 t wherein the restoring step includes restoring the document to the 
original state by removing all of the different watermarks in accordance with an embedding 
sequence. 

Although Rhoads discloses embedding multiple watermarks in the same image (Section 2 
page 2068), Fridrich, Wong, and Rhoads do not expressly disclose restoring the document to the 
original state by removing all of the different watermarks. 

At the time the invention was made, it would have been obvious to a person of ordinary 
skill in the art to remove the multiple watermark to restore the document to the original form in 
the system of Fridrich. One of ordinary skill in the art would have been motivated to do this 
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because the watermarks add noise to the image and therefore to restore the image to the original 
form the noise would need to be removed. 

Conclusion 

Any inquiry concerning this communication or earlier communications from the 
examiner should be directed to Paula W Klimach whose telephone number is (703) 305-8421. 
The examiner can normally be reached on Mon to Thr 9:30 a.m to 5:30 p.m. 

If attempts to reach the examiner by telephone are unsuccessful, the examiner's 
supervisor, Kim Vu can be reached on (703) 305-4393. The fax phone number for the 
organization where this application or proceeding is assigned is 703-872-9306. 

Information regarding the status of an application may be obtained from the Patent 
Application Information Retrieval (PAIR) system. Status information for published applications 
may be obtained from either Private PAIR or Public PAIR. Status information for unpublished 
applications is available through Private PAIR only. For more information about the PAIR 
system, see http://pair-direct.uspto.gov. Should you have questions on access to the Private PAIR 
system, contact the Electronic Business Center (EBC) at 866-217-9197 (toll-free). 

The 2100 Tech center will move to Carlyle in October 2004. The new telephone number 
for the receptionist is (571) 272-2100. The examiner's new telephone number will be (571) 272- 
3854. 

Friday, September 17, 2004 /L-~\ /I 
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IF ONE WATERMARK IS GOOD, ARE MORE BETTER? 



Fred Mintzer and Gordon W. Braudaway 

IBM TJ. Watson Research Center 
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ABSTRACT 

Invisible watermarts are not all alike. Different techniques arc 
used to embed different types of watermarks into digital media 
objects to accomplish different goals. Some watermarks are 
intended to robustly carry ownership information; some are 
intended to carry content-verification information; and some are 
intended to convey side information, or captions. In this talk, 
some opportunities to employ multiple watermarks to convey 
multiple sets of information, intended to satisfy differing or 
similar goals, are examined. Problems presented by the insertion 
of multiple watermarks are discussed. Progress towards 
developing techniques that embed multiple watermarks into an 
image will also be presented. 



1. INTRODUCTION 

Invisible watermarking involves imperceptibly embedding data, 
called a watermark, into an image or other digital media object to 
enhance or protect its value. Although watermarking is relatively 
new field, many applications have already been proposed. 
Recent work (1.2,3} has identified three clusters of applications. 
One uses watermarks to convey ownership information. A 
second uses watermarks to verify that object content has not 
changed. A third uses watermarks to convey object-specific 
information, or captions, to a community of willing recipients 

The application clusters generally require application-specific 
watermarking techniques to best address them. In the remaining 
subsections of this Introduction, we briefly describe the 
application clusters and summarize their technical requirements. 

1.1 Watermarks to convey ownership info 

Applications thai convey ownership information are often desired 
by organizations thai own the copyrights to digital media objects 
and license them. These organizations include news agencies 
and photo banks, museums and libraries. 

In a typical image application, the content owner provides an 
image thai will be published by the recipient; in exchange, the 
owner will receive a royalty for the image's use. A concern of the 
owner is that the publisher will neglect to pay the royalty; this 
omission may be in ten dona) or unintentional. To deter such 
misappropriation, the owner may wish to place a watermark, 
containing ownership information, in the image. 

The ownership information may identity either the owner or the 
recipient If the watermark identifies the owner, the owner might 
subsequently scan suspect published material to determine 
whether a printed image contained his watermark; the owner 



would consider its presence to be evidence of his ownership. If 
the watermark contained the recipient's identification, the owner 
might subsequently scan the published material to determine who 
received the material; if the image had been used without 
payment of royalties, the owner might wish to cease doing 
business with the recipient. Descriptions of three scenarios that 
use invisible watermarking to convey ownership information are 
given in [2]. Many techniques for applying an ownership 
watermark are described in [4}. 

For a watermark to be effective in ownership applications, it 
must remain in the media object and be reliably detectable from 
the published product. For the publishing example described 
above, it is important that the watermark be reliably detectable 
from the scan of a published image. 

This is a considerable challenge. Before printing, an image is 
often cropped, re-sized, sharpened, contrast enhanced, color 
corrected, and JPEG compressed. Hence, a watermark for 
ownership applications must survive a variety of image 
processing tasks that are routinely used to prepare images for 
publishing. Furthermore, there is an economic incentive for 
publishers to remove watermarks. Those publishers that would 
seek to not pay royalties might attempt to remove the watermark, 
and many common image processing tools are available to 
accomplish this. Thus, for an ownership watermark to be 
effective it should robustly resist image-processing attacks, 
intentional and unintentional, that might remove it. For that 
reason, we describe watermarks for ownership applications as 
robust watermarks. 

One straightforward attack can be mounted whenever a malicious 
party has access to a watermark in the clear, where a watermark 
"in the clear" is one that can be extracted by anyone who has 
access to the extraction process. With this attack, the attacker 
alters the watermarked image and extracts the watermark. If the 
inserted watermark is identical to the extracted watermark, a 
different or additional alteration is attempted. The sequence of 
alterations continues until the extraction process reveals that the 
mark has been removed or altered; then, the evidence of 
ownership has been removed. To counter this iterative attack on 
watermarks in the clear* most ownership marks are inserted with 
a watermark key. In this case, the watermark key is required to 
extract the watermark and that key is only supplied as needed. 
Without the watermark key, the attacker cannot extract the 
watermark with this attack. We note that the phase "in the clear" 
is now generally used to describe watermarks mat are not 
protected by a watermark key. 

Three recent papers [5,6,7] have discussed attacks to remove 
ownership watermarks. They provide a more detailed description 
of what watermark robustness entails. 
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Robust watermarks are often applied with great redundancy to 
achieve the robustness desired. Hence, ownership watermarks 
generally convey relatively small amounts of data, ranging from 
tens to hundreds of bits. 

1.2 Watermarks to verify that object content 

A second application cluster uses watermarks to determine 
whether a media object has been altered since some earlier time 
when it was watermarked. Some applications scenarios for this 
cluster are also given in [2]. Some example techniques are 
described in [9,10,11]. 

In a typical image application, an image is watermarked at the 
time it is loaded into a digital library. At some later rime, the 
watermark is extracted. If the extracted watermark matches the 
inserted watermark, the object is judged to be unchanged; if it 
does not match the inserted watermark, the image is judged to 
have been altered. An inspection of the extracted watermark 
(and its difference from the inserted watermark) reveals where 
the alterations have occurred. For these applications, it is desired 
that the watermark will be altered if the image is altered; we call 
these watermarks fragile watermarks because they should be 
easily damaged if the object is altered. 

If a fragile watermark is in the clear, one might alter the content 
of a false image, extract its watermark* and interate until the 
extracted watermark matched the watermark inserted in the true 
image. To prevent such a false insertion, a watermark key is also 
often used with fragile watermarks. 

Fragile watermarks do not require redundancy. Consequently, 
they often convey very large amounts of data. The technique of 
[11]. for example, embeds one bit of watermark data for each 
pixel in the marked image; this can be millions of bits. 

1.3 Watermarks to convey captions 

The third application cluster uses watermarks to convey object- 
specific information, called captions, to a community of willing 
recipients. With these applications, both the content owners and 
the recipients desire that the information be conveyed. 

In a typical application, a digital photo agency uses a watermark 
to convey data that records the name of the owning agency and 
an image identification number. Then, when the image is 
published, the publisher knows whom to contact for permission. 
Furthermore, the owning agency knows how much to charge and 
what photographer to compensate (since it knows which image 
is being used). Other proposed collaborative watermarking 
applications are described in [1]. 

To enable proper conveyance, collaboration among the 
community of content owners and the community of recipients is 
needed. This is done so that the content owners may employ a 
single watermarking technique and the recipients can employ a 
single watermark extraction technique. Without such standards, 
it is difficult to imagine how this class of applications could be 
undertaken. 

Although the recipients desire to extract the watermark 
information, some robustness is required, as captioning 
watermarks should survive unintended attacks to be most useful. 



However, captioning watermarks must often convey more 
information than ownership watermarks. Consequently they can 
employ less redundancy, and hence they are less robust. 

2. Multiple Watermarks to Address Multiple 
Applications 

To accomplish several goals, one might wish to embed several 
watermarks into the same image. For example, the owner might 
desire to: 

• use one watermark to convey ownership information, 

• use a second watermark to verify content integrity, and 

• use a third watermark to convey a caption (that might 
describe the content of the image). 

In attempting to accomplishment this, one should first look at the 
robustness of each technique. As noted earlier, 

• ownership watermarks should be very robust, 

• captioning watermarks should be robust, and 

• verification watermarks should be quite fragile. 

Embedding a fragile watermark followed by embedding a robust 
watermark is bound to damage the fragile watermark. Indeed, by 
design, a fragile watermark should be damaged by any operation 
that alters the image, and robust watermarking is such an 
operation. 

In general, to apply multiple disparate watermarks, 

• the most robust (ownership) watermark should be embedded 
first, 

• the most fragile (verification) watermark should be 
embedded last, and 

• moderately robust (captioning) watermarks should be 
inserted in between. 

Embedding multiple watermarks will then be successful if the 
robust watermarks are sufficiently robust to withstand all 
subsequent watermark insertions 

After the insertion of multiple watermarks, the watermarked 
image will posses texture resulting from each watermark. 
Embedding multiple watermarks, we note, also requires that each 
watermark add less texture than would be permissible if that 
watermark were employed alone. 

Hence, embedding multiple disparate watermarks to an image 
can be successful if 

• the robust watermarks are sufficiently robust, and 

• each invisible watermark is sufficiently sub-visible that the 
suite of watermarking is still invisible. 
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3. Multiple Watermarks to Address one 
Application Multiple Times 

One might also envision embedding multiple watermarks for 
similar applications. 

3.1 Multiple ownership watermarks 

For example, as noted earlier, ownership watermarks are 
sometimes used to identify the owner and sometimes used to 
identify the recipient. One can imagine employing two 
ownership watermarks to identify both the owner and the 
recipient 

Here, the question arises. How should the two watermarks best 
be embedded? Intuitively, we expect that embedding them 
simultaneously is desired. This is a topic of current research. 

3.2 Multiple verification watermarks 

One might envision employing multiple verification watermarks 
to an image to increase verification security. Then, if party one, 
possessing key one, forged a watermark on a false image, party 
two, using key two, would still be able to detect the alteration. 

Since verification watermarks are intentionally fragile, one 
cannot apply a second fragile watermark after the first. Hence, to 
apply two verification watermarks, one must apply them 
simultaneously. How to accomplish this is another topic of 
current research. 

3.3 Multiple watermarks for multiple captions 

One might imagine the use of multiple captioning watermarks to 
apply multiple captions in the case where a recipient wants to 
add supplemental captioning to an object. 

Here, the recipient might merely extract the original caption, 
append an additional caption, and embed a watermark that 
conveys the expanded caption. 

However, if this were the case, the image would have been 
degraded by the application of two captioning watermarks. It 
might simply be more effective to contact the owner and ask that 
a watermark, conveying the expanded watermark, be applied to 
the original image. Hence, the need for multiple captioning 
watermarks is not readily apparent. 

4. Remarks 

Above, we have envisioned several situations in which it might 
be desired to embed multiple watermarks in an image. 

When the multiple watermarks are intended to satisfy different 
applications, the order in which the watermarks are applied is 
vital. Increased robustness of the robust watermarks and 
decreased visibility of all watermarks are desired in this 
environment. 

In the case where multiple watermarks of the same type are 
embedded, it may be advantageous to develop techniques that 
apply multiple watermarks simultaneously. This is especially 



true for verification watermarks; in this case, the multiple 
watermarks must be applied simultaneously. 
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